Cooperative adaptive cruise control: A reinforcement learning approach,”IEEE

Abstract—Recently, improvements in sensing, communicating, and computing technologies have led to the development of driver-assistance systems (DASs). Such systems aim at helping drivers by either providing a warning to reduce crashes or doing some of the control tasks to relieve a driver from repetitive and boring tasks. Thus, for example, adaptive cruise control (ACC) aims at relieving a driver from manually adjusting his/her speed to maintain a constant speed or a safe distance from the vehicle in front of him/her. Currently, ACC can be improved through vehicle-to-vehicle communication, where the current speed and ac-celeration of a vehicle can be transmitted to the following vehicles by intervehicle communication. This way, vehicle-to-vehicle com-munication with ACC can be combined in one single system called cooperative adaptive cruise control (CACC). This paper investi-gates CACC by proposing a novel approach for the design of au-tonomous vehicle controllers based on modern machine-learning techniques. More specifically, this paper shows how a reinforce-ment-learning approach can be used to develop controllers for the secure longitudinal following of a front vehicle. This approach uses function approximation techniques along with gradient-descent learning algorithms as a means of directly modifying a control pol-icy to optimize its performance. The experimental results, through simulation, show that this design approach can result in efficient behavior for CACC. Index Terms—Autonomous vehicle control, cooperative adap-tive cruise control (CACC), neural networks, policy-gradient algorithms, reinforcement learning (RL). I